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FOREWORD 


This  research  was  performed  by  the  Human  Resources  Research  Organi¬ 
zation  (HumRRO).  The  study  was  initiated  as  technical  advisory  service 
and  was  completed  as  part  of  Work  Unit  STOCK,  Development  of  Training 
Management  Procedures  for  Different  Ability  Groups.  The  research  was 
completed  while  HumRRO  was  part  of  The  George  Washington  University. 

This  report  provides  a  description  of  (a)  the  procedures  used  to 
analyze  Supplyman  duties  and  tasks  to  derive  a  set  of  training  objectives, 
including  standards  of  performance,  (b)  techniques  by  which  training 
content  was  aligned  with  established  course  objectives,  (c)  the  interim 
comparison  of  Pilot  and  Army  Training  Center  courses,  and  (d)  the  final 
evaluation  of  the  revised  Supplyman  course. 

The  study  was  performed  under  the  direction  of  Dr.  A.  James  McKnight, 
the  Work  Unit  Leader.  The  task  analysis  was  supervised  by  Dr.  Harold 
Wagner,  assisted  by  Mr.  James  R.  Lodge,  HumRRO  and  MSG  Harry  G.  Abel,  the 
Quartermaster  School,  with  the  participation  of  other  military  and  civil¬ 
ian  personnel  of  the  Quartermaster  School .  The  computer  program  was 
written  by  Mrs.  Bettye  Boggs  and  the  processing  performed  by  HumRRO 
Computer  Center  personnel.  Dr.  Richard  D.  Behringer  developed  and  admin¬ 
istered  the  end-of-course  performance  tests.  The  discrepancy  analysis 
was  performed  by  Di .  Wagner.  From  the  results  of  the  discrepancy  analysis, 
SP-5  Donald  Quigley  revised  the  task  analysis  document.  Mrs.  Jane  V.  Lee 
organized  and  revised  the  final  report. 

HumRRO  research  for  the  Department  of  the  Army  is  conducted  under 
Contract  DAHC  19-70-C-0012 .  Training  Motivation,  and  Leadership  research 
is  conducted  under  Army  Project  2Q062107A712 . 
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SUMMARY  AND  CONCLUSIONS 


PROBLEM 

In  19t>0,  the  Department  of  Defense  instituted  Project  100,000,  a 
program  which  reduced  the  prevailing  mental  and  physical  standards  for 
the  armed  services  to  permit  the  annual  induction  of  an  additional 
100,000  men  each  year.  This  program  included  a  variety  of  activities 
intended  to  facilitate  the  absorption  of  these  "New  Standards"  personnel 
by  the  services.  Redesigning  training  programs  to  accommodate  New 
Standards  personnel  was  among  these  activities. 

The  U.S.  Army  Quartermaster  School  at  Port  Lee,  Virginia,  requested 
UumRRO  assistance  in  evaluating  its  revision  of  the  Supplyman  (MOS  "n.MOl 
course.  This  training  program  had  been  designated  as  a  pilot  course  to 
pioneer  techniques  for  dealing  with  the  New  Standards  input. 

OBJECTIVES 

The  purpose  of  this  study  was  to  evaluate  the  effectiveness  of  the 
revised  Supplyman  course  in  accommodating  New  Standards  personnel  without 
a  decline  in  performance  standards. 

APPROACH 

A  comprehensive  analysis  of  the  Supplyman's  duties  and  tasks  was 
performed  in  order  that  appropriate  training  objectives  and  performance 
standards  could  be  specified.  A  task  analysis  technique,  amenable  to 
computer  processing,  was  developed  and  provided  data  on  a  continuing  basi 
to  curriculum  developers.  In  addition,  a  quality  control  procedure  was 
instituted  for  checking  course  content  against  the  task  analysis. 

The  focus  of  this  effort  was  the  construction  of  an  end -of- course 
performance  measure.  Supply  tasks  were  selected  for  a  job  sample  perform 
ancc  test  which  was  administered  to  graduates  of  the  pilot  course  and  its 
conventional  counterpart  at  a  U.S.  Army  training  Center  (.ATC)  for  an 
interim  evaluation.  The  performance  test  was  then  modified  and  adminis¬ 
tered  to  graduates  of  the  revised  pilot  course  to  evaluate  their  profi¬ 
ciency  i it  meeting  performance  standards  derived  from  the  task  analysis 
(final  evaluation!. 

RESULTS 

The  interim  evaluation  demonstrated  the  following: 

(.1)  Kith  the  exception  of  non-criticul  errors  and  typing. 

Category  1 1  - 1 1 1  personnel  scored  significantly  higher  on  all  tests  than 
Category  IV  personnel. 
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(. - )  The  pilot  course  graduates  required  more  time  and  committed 
more  critical  errors  than  the  A'lV  graduates  on  the  performance  test. 

(,31  The  pilot  group  required  less  assistance  and  committed  fewer’ 
non-critical  errors  than  the  Alt'  group  on  the  performance  test. 

(41  The  All"  group  performed  significantly  better  than  the  pilot 
group  on  tests  of  reading,  arithmetic,  and  typing. 

(51  The  pilot  group  surpassed  the  ATt'  group  on  the  supply  knowledge 

test . 

(C>1  There  was  no  attrition  or  recycling  of  pilot  course  students, 
whereas  10  percent  of  Alt'  students  were  dropped  and  nearly  one  quarter  of 
the  sample  used  in  this  study  had  been  recycled. 

The  final  evaluation  of  the  revised  pilot  course  indicated  the 
t'ol  low  ing : 

(11  On  the  test  as  a  whole,  the  class  performed  over  SO  percent 
of  the  task  elements  correctly. 

(~)  Students  did  not  perform  any  more  reliably  on  highly  critical 
items  (.8  and  .91  than  they  did  upon  less  critical  (.5  and  .'1  items.  As 
a  result,  they  failed  to  meet  performance  requirements  of  the  former  while 
exceeding  those  of  the  latter. 

CONCLUSIONS 

The  findings  of  the  interim  evaluation  were  of  limited  value  in 
comparing  the  pilot  course  against  the  A IV  course,  because  of  different 
emphases  placed  on  the  curricula,  and  differences  in  student  samples  and 
test  administrations.  However,  the  Project  100.000  goal  of  accommodat ing 
new  standards  personnel  without  reducing  performance  standards  was 
achieved,  as  evidenced  by  the  reduction  in  attrition  achieved  by  the 
pilot  group  in  comparison  to  that  of  the  A IV  program,  without  significant 
performance  decrement.  While  each  group  exceeded  the  other  on  certain 
tests,  the  balance  favored  neither  group. 

Although  overall  performance  was  good,  the  final  evaluation  indicated 
that  graduates  of  the  revised  pilot  course  failed  to  attain  the  standards 
of  performance  set  for  highly  critical  items.  As  these  standards  were 
arbitrary,  it  is  possible  that  they  were  set  unrealistically  high.  It  is 
also  possible  that  insufficient  emphasis  was  placed  on  the  critical  areas 
in  the  course.  Evidence  indicates  that  instructors  did  not  have  enough 
time  to  use  the  task  analytic  informat  ion  when  determining  which  areas  to 
stress  in  the  course.  Overall,  however,  the  revised  course  developed  by 
the  Quartermaster  School  effectively  accommodated  New  Standards  personnel, 
thus  achieving  its  objective. 


Chapter  1 

DETERMINATION  OF  TRAINING  OBJECTIVES 


INTRODUCTION 

In  October  19oo,  the  Secretary  of  Defense  instituted  a  program  by 
which  prevailing  mental  and  physical  standards  for  the  armed  services 
were  reduced  to  permit  the  annual  induction  of  an  additional  100,000  men 
each  year.  Approximately  SSS  of  these  "New  Standards"  inductees  would 
enter  under  the  lowered  mental  standards  and  the  majority  of  them  -  over 
b0,000  --  would  be  assigned  to  the  Arms. 

Project  100,000,  as  the  program  was  called,  included  in  addition  to 
the  New  Standards  personnel,  a  variety  of  activities  intended  to  facili¬ 
tate  their  absorption  by  the  armed  services.  Among  these  efforts  was 
the  redesign  of  military  training  programs  to  enable  them  to  better 
accommodate  personnel  of  lowered  academic  promise.  Several  Army  programs 
were  designated  pilot  courses  to  pioneer  techniques  for  dealing  with  the 
New  Standards  input.  Among  these  was  the  Supplyraan  (MOS  "oAlOi  course 
for  which  the  U.S.  Army  Quartermaster  School  iQMSl  at  Port  lee,  Virginia, 
had  proponencv. 

A  request  was  made  by  the  QMS  for  the  assistance  of  llumRKO  in  evalu¬ 
ating  the  results  of  the  Supplyman  course  revision.  The  measure  of 
success  was  to  be  the  ability  of  the  pilot  course  to  meet  the  Project 
100,000  goal  of  accommodating  New  Standards  personnel  without  a  decline 
in  the  standards  of  performance.  HumRKO  part icipat ion  commenced  in  May 
of  19b" . 

The  focus  of  this  HumKRO  effort  was  construction  of  an  end-of-eourse 
performance  measure  to  be  administered  to  graduates  of  the  pilot  classes. 
In  order  to  achieve  this  objective,  performance  spec i f icat ions  were 
needed  by  which  to  test  the  students  and  thus  evaluate  the  revised 
Supplyman  course.  A  task  list  had  been  prepared  by  the  Project  100,000 
group  at  the  Quartermaster  School .  lhis  list  served  at  the  basis  for  the 
first  revision  of  the  Supplyman  course.  There  were,  however,  deficiencies 
in  the  original  task  list  due  primarily  to  the  short  period  of  time  that 
QMS  had  available  for  its  development.  It  did  not  identify  all  of  the 
tasks,  nor  did  it  describe  in  sufficient  detail  how  they  would  be  accom¬ 
plished.  For  example,  instead  of  "The  Supplyman  assists  his  supervisor,” 
the  precise  responsibilities  of  each  individual  needed  to  be  identified. 
The  task  analysis  technique  to  be  described  was  developed  to  identify  all 
relevant  tasks,  skills,  knowledges  and  performance  standards  required  of 
the  Supplyman.  The  results  of  this  analysis  were  used  in  a  further 
revision  of  the  pilot  course  as  well  as  in  the  construction  of  an  eiul-of 
course  performance  test . 


DEFINITION  OF  "TASK 


One  of  the  difficulties  in  the  field  of  task  analysis  is  the  lack  of 
agreement  regarding  definition  of  the  term  "task."  In  the  literature 
"task"  definitions  range  from  the  highly  specific  to  the  al 1 -encompass ing . 
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a. 

"The  spec  it: 

to  action  t 

aken  bv  an.  tndivi 

dual  in 

performing 

his  duty.  The  task  has  identifiable  starting  and  ending 
points  and  results  in  a  measurable  product."  4 

"A  collection  ot  activities  that  are  performed  by  one 
person,  bounded  by  two  events  and  deseribable  so  that 
the  resulting  task  description  conveys  enough  intormation 
about  the  task  to  permit  the  necessary  training  decisions 
to  be  made."  5 


The  HumRRO  task  analysis  of  the  Supplyman  job  was  focused  primarily 
upon  training.  Since  certain  characteristics  of  a  task  are  important  for 
training  purposes,  these  were  selected  in  describing  the  behavioral 
events  --  such  characteristics  as  the  context  in  which  the  activity  occurs 
and  the  performance  components  of  the  activity  itself.  The  definition  of 
"task"  underlying  the  task  analysis  technique  employed  in  this  study  was 


lShriver,  F.L.  .4  The-evei L\il  dprivaoc  re  .".uv.xtsr  ;he  TV. j 
Dtmande  T-::\\u\:  bu  .We  4 .Were:  oh, a ; < -.'c .  Staff  Memorandum,  HumRRO 
Division  No.  1  (System  Operations'),  Alexandria,  Virginia,  December  195b. 
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Technical  Report  TDR  b5-Jl,  Behavioral  Sciences  Laboratory,  Aerospace 
Medical  Research  Laboratories,  Air  Force  Systems  Command ,  Kright-Patterson 
AFB,  Ohio,  January  19b4  (Contractor:  Radio  Corporation  of  .America). 
3Darby,  C.L.,  Brown,  K.F.,  Smith,  C.D.,  and  Fightmaster,  IV. J.  The 
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Headquarters,  USCONARC,  Fort  Monroe,  Virginia,  February  19bS. 
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a  combination  of  definitions  d  and  e: 

Task  -  The  specific  action  taken  by  an  individual  in 

performing  his  duty.  The  task  has  identifiable 
starting  and  ending  points  and  results  in  a 
measurable  product.  The  description  of  the  task 
is  directed  toward  providing  information  on  which 
training  content  can  be  based. 

PROCEDURES 

To  provide  course  developers  with  the  necessary  information  for  making 
training  content  decisions,  the  task  analysis  attempted  to  provide  a 
total  task  inventory,  listing  all  the  possible  tasks  that  a  job  incumbent 
may  perform.  It  became  apparent  early  in  the  stud)’  that  the  Supplyman 
could  be  assigned  to  a  great  variety  of  duty  positions  in  many  types  of 
units.  Continuing  the  development  of  a  total  task  inventory  would  have 
been  impractical  because  there  would  not  be  time  to  train  the  Supplyman 
in  all  tasks.  Therefore,  as  task  lists  were  compiled  they  were  forwarded 
to  QMS  for  decisions  as  to  which  tasks  would  be  taught  in  the  76A10 
course.  If  a  task  was  to  be  taught,  it  was  analyzed;  if  not,  it  was 
deleted  from  the  analysis  The  final  task  list  thus  reflected  the 
training  objectives  selected  by  QMS. 

Categorization  of  Supplyman  Activities 

The  activities  that  individuals  with  the  76A10  MOS  are  likely  to 
perform  were  separated  into  two  broad  supply  process  level  categories 
--  Unit  and  Organization  (U  &  0) ,  and  Support.  The  latter  category  was 
further  subdivided  into  Stock  Control  and  Storage  procedures. 

A  matrix  was  constructed  to  categorize  the  Supply-man's  U  §  0  tasks. 

This  matrix  consisted  of  a  list  of  "actions"  along  one  dimension  and  the 
"objects"  of  these  actions  (supplies)  along  the  other  dimension.  The 
actions  were  (1)  receiving  requests  for  supplies,  (2)  making  requests, 

(3)  receiving  supplies,  (4)  issuing  supplies,  (5)  receiving  turn-ins, 
and  (6)  making  turn-ins  to  higher  support  activities.  These  action 
categories  were  inclusive  enough  to  describe  the  majority  of  tasks  which 
the  76A10  would  perform  at  the  U  5  0  level.  The  objects,  i.e.,  the 
commodities  upon  which  these  actions  would  be  taken  included  (1)  individual 
clothing,  (2)  organizational  clothing  and  equipment,  (3)  non-expendable 
supplies,  (4)  repair  parts,  (5)  other  expendable  supplies,  t6)  petroleum, 
oils  and  lubricants  (POL),  and  (7)  rations.  This  produced  a  6  x  7  matrix 
containing  42  cells. 

The  matrix  was  refined  through  a  series  of  structured  interviews  with 
QMS  content  experts.  The  primary  objectives  of  these  interviews  were  to 
determine  which  tasks  delineated  by  the  matrix  were  actually  performed  by 
the  Supplyman  and  what  steps  were  involved  in  performing  each  task. 

During  the  initial  session,  those  cells  were  eliminated  which  either  were 
not  U  §  0  supply  functions  or  whose  elements  did  not  differ  from  the  other 
cells  within  the  matrix  and  could  be  subsumed  under  them.  Additional 
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U  ti  0  tasks  not  covered  by  the  matrix  wore  listed  separately  (e.g., 
handling  of  laundry,  requesting  maintenance,  etc.). 

The  Support  level  tasks  were  categorized  differently  from  the  U  ti  0  ' 
tasks.  In  support  units,  an  individual  Supplyman  is  likely  to  be  respon¬ 
sible  for  a  restricted  group  of  tasks  whereas  in  the  U  ti  0  supply  room  it 
is  highly  probable  that  a  single  individual  would  perform  the  majority  of 
the  supply  tasks.  Support  level  tasks  were  thus  categorized  according  to 
duty  positions,  such  as  document  receiving  clerk,  stock  accounting  clerk, 
etc . 

The  interview  sessions  occurred  over  a  period  of  several  weeks  during 
which  a  content  expert,  assuming  the  role  of  a  Supplyman,  outlined  the 
steps  involved  in  performing  the  tasks.  This  information  was  recorded 
and  compared  witli  existing  doctrine  and  supply  publications.  Inconsist¬ 
encies  were  resolved  through  further  discussion  with  QMS  personnel. 

In  this  manner,  listings  of  tasks  and  their  components  were  prepared. 
As  the  task  analysis  progressed  a  coordination  system  was  set  up  between 
lluinRRO  and  QMS  in  order  that  decisions  regarding  the  inclusion  of  tasks 
in  the  course  would  be  made  available  to  the  task  analysts.  The  proce¬ 
dure  enabled  the  task  analysts  to  interact  with  the  training  program 
developers  during  the  time  of  curriculum  development  and  to  have  current 
information  continuously  available.  Also  through  this  interaction,  the 
analysts  provided  the  training  program  developers  with  information 
necessary  for  realistic  job-oriented  training.  This  method,  however, 
necessitated  numerous  revisions  in  the  task  list.  To  accommodate  these 
revisions  within  the  allotted  timeframe  would  have  been  impossible  without 
the  aid  of  a  computer.  A  task  analytic  technique  was  developed  in  which 
computer  processing  could  be  utilized  for  recording  tasks  and  their 
components . 

Computer  Processing 

The  need  for  computer  assistance  in  handling  the  enormous  volume  of 
task  analytic  data  that  can  arise  within  the  military  occupational  struc¬ 
ture  can  be  seen  in  the  recent  research  and  development  efforts  by  the 
Air  force1 and  by  the  U.S.  Naval  Personnel  Research  Activity3,  and  in 


1  Pot  ter,  K.W.,  Till  ley,  A.T.,  and  Reed,  I..T.  /V;v.’.  Apj'i  i>\i- 

1 1 on  e/  7V. tv  !-\ivt  wv  t'v.vk  Iivn.il  ■  n.j 

I'n'b l ■. Technical  Report  AMRL-TR-oo- JOO  ,  Aerospace  Medical  Research 
Laboratories,  Wright -Patterson  APR,  Ohio,  December  19oo. 

2Tulley,  A.T.  and  Meyer,  d.R.  1-ifi  v’.vntvt . ■/  V. :•  bvftuW. 

v  t  v  iliu'i.v:  i-vv/vw  T.ivk  IKit.i  it  ,v:.iii>:v.  i'vvb  i  v:v .  Technical  Report 
AMRL-TR-07- 1 27 ,  Aerospace  Medical  Research  Laboratories,  Wright  Patterson 
APR,  Ohio,  September  li)o7. 

'Campbell,  G.M.  /I  .'  /  .vi-uir.i  I A*  b\  t  /.  .;Viv,  a.  ;'r.  - 

"r  ,'ntv  InJ'vf'Uit  :vn  .V,  i  ■  :<  7 : 7.' 1 .  /  c. :s‘v  .V.  .  Re¬ 

search  Memo.  SRM  (wS-17,  U.S.  Naval  Personnel  Research  Activity,  San  Diego, 
California,  March  lDoS. 


the  work  being  done  on  the  Military  Occupational  Information  Data  Hank 
under  the  auspices  of  t ho  Office  of  Personnel  Operations,  Department  of 
the  Army . 

To  prepare  the  task  list  for  computer  processing,  recording  forms 
were  developed  fsec  Appendix  A)  from  winch  the  information  could  be 
prepared  for  keypunching  and  eventual  storage  in  the  computer.  Also,  a 
program  was  developed  with  the  assistance  of  the  llumKRO  Computer  Center 
for  computer  printing  of  this  information  whenever  needed. 

Recording  forms.  The  type  of  information  that  was  recorded  on  the 
forms  is  described  in  fables  1  and  7.  Recording  lorm  Card  1  contained 
columns  for  an  identifying  number,  task  or  task  component  statement,  and 
a  project  designator  code.  The  identifying  number  reflected  t ho  hierar 
chical  level  of  the  task  description.  The  first  digit  indicated  the 
supply  process  level  (U  t*  0,  Stock  Control  or  Storage).  The  second  and 
third  digits  indicated  the  activity  or  duty,  and  the  major  task,  for 
example,  at  the  0  8  0  level,  t he  duty  could  be  "receiving  supplies"  and 
the  major  task  "receiving  repair  parts."  The  remaining  four  digits 
represented  a  hierarchical  arrangement  of  task  components  ii.e.,  subtasks, 
elements).  The  task  description  section  of  the  form  provided  a  space  to 
record  a  statement  describing  the  task  or  task  component .  Abbreviations 
were  used  to  permit  inclusion  of  all  t he  information  on  one  punch  card. 


Table  1 


ENTRIES  ON  TASK  ANALYSIS 

Recording  form  Columns 
1-7 

1 

•I-" 

8 

9  -  70 

7t>-  79 


RECORDING  TORM  -  CARD  1 


Informal  ion  Recorded 

Identifying  number 

level  of  supply  (1  -  Unit 
5  Organ i rat  ion ,  2  -  Stock 
Control,  A  -  Storage) 

Task 

Task  components 
blank 

Task  or  task  component 
descript  ion 

blank 

Project  designator  code 


80 


Card  number  "1" 


Table  2 


ENTRIES  ON  TASK  ANALYSIS  RECORDING  FORM  -  CARD  2 


Recording  Form  Columns 
1-7 

8 

9-20 

9- ID 
1 1  -  20 

2 1 

JJ-So 

S' 


In format  ion  Recorded 
Identifying  number 

bl  ank 

I'oet  rine  reference 
Reference  code 
Page  number 

blank 

knowledge  descript  ion 
blank 


58 


Skill  code  (1  -  perceptual, 
-  motor,  -  cognitive) 


5  Si 


blank 


oO 


Criticality  judgment  perfonn 
attce  standard 


Ol 


bl  ank 


oJ-oo 

(>4-05 


Course  location 
Page  numbers 
lesson  plan  code 


oo 


b  l  ank 


O' -08 


braining  objectives  (not  used 
in  analysis) 


oP  -  - 


b  l  ank 


7 0  -  'll 


Project  designator 


80 


Card  number 


Task  -  related  information  useful  for  course  development  and  evaluation 
was  entered  on  Recording  form  Card  .  following  the  seven  digit  idem  i 
Tying  number  mentioned  above,  a  field  labeled  "Reference"  was  provided. 
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This  entry  consisted  of  a  code  number  th;it  pertained  to  the  specific 
doctrinal  reference  for  that  task  and/or  component ,  followed  by  the  page 
numbers  within  the  document  that  related  to  the  particular  item  so  that 
changes  in  doctrine  could  be  rapidly  located. 

Following  the  reference  entry,  space  was  allocated  for  the  insertion 
of  a  "Knowledge  Description."  Statements  describing  the  supplementary 
knowledge  necessary  to  enable  the  Supplvman  to  perform  the  task  or 
component  were  entered  in  this  space.  Abbreviations  were  used  whenever 
space  constraints  made  it  necessary.  This  was  followed  by  a  one-letter 
code  which  indicated  the  category  of  the  enabling  skill  (perceptual, 
motor,  or  cognitive)  necessary  for  task  performance.  Relating  knowledge 
and  skill  requirements  to  specific  tasks  provides  a  basis  for  insuring 
that  each  item  of  the  training  content  is  related  to  some  aspect  of  job 
performance . 

The  next  field  of  Recording  Form-Card  J  was  labeled  "Criticality." 

This  term  refers  to  the  importance  of  particular  tasks  to  the  supply 
process.  Judgments  were  made  by  supply  experts  regarding  the  importance 
to  the  supply  system  of  errors  in  the  performance  of  these  items.  Those 
judgments  were  later  translated  into  standards  of  performance  necessary 
for  graduation  in  the  7oA10  course.  Hach  component  of  every  task  was 
judged  individually,  here  standards  to  have  been  assigned  to  a  task  as 
a  whole,  no  distinction  would  have  been  made  among  the  different  errors 
that  can  occur  in  the  specific  elements  that  constitute  task  performance. 
For  example,  "Must  complete  DA  Form  Mol  with  no  more  than  three  errors" 
does  not  distinguish  between  an  error  in  typing  a  Federal  Stock  Number 
and  an  error  in  typing  someone's  name,  yet  the  former  is  far  more  critical. 
By  assigning  standards  to  individual  task  elements,  performance  can  be 
evaluated  more  realistically  and  meaningfully.  Therefore,  with  the  aid  of 
content  specialists  from  the  various  branches  of  the  Fit  listed  Supply 
Department,  standards  of  performance  were  assigned  to  the  categories  of 
criticality,  as  follows: 

Category  1  -  F.rrors  that  would  result  in  no  supplies,  the  wrong 
supplies,  or  severe  delays  in  the  processing  of  supplies.  The  standard 
of  performance  necessary  for  graduation  assigned  to  these  items  was  DO 
percent . 

Category  2  -  F.rrors  that  would  result  in  the  wrong  quantities  or 
minor  delays  in  the  processing  of  supplies.  The  assigned  standard  of 
performance  was  80  percent . 

Category  a  -  1  rrors  that  could  have  an  effect  on  the  processing 
of  supplies  but  would  probably  be  detected  and  corrected  locally.  The 
assigned  standard  ot  performance  was  "0  percent  . 

Category  1  F.rrors  that  would  not  have  an  effect  on  the 
processing  of  supplies  and  would  primarily  be  due  to  carelessness.  Hie 
assigned  standard  of  performance  was  NO  percent. 


Although  one  cannot  make  a  strong  case  as  to  the  validity  of  these 
standards,  since  they  were  arbitrarily  determined  for  use  as  a  frame  of 
reference,  they  provided  a  starting  point  and  were  capable  of  revision 
should  the  need  have  arisen. 

The  "Course  Location"  section  contained  entries  which  reflected  the 
position  of  each  of  the  tasks  and/or  components  within  the  lesson  plans 
of  the  76A10  course.  The  first  two  digits  reflected  the  page  number  at 
which  instruction  began  on  a  specific  item  and  the  second  two  digits,  the 
lesson  plan  in  which  the  item  was  being  taught. 

Task  Analysis  Document.  A  task  analysis  that  enters  the  curriculum 
development  process  is  subject  to  frequent  revision  in  response  to 
addition  and  deletion  of  tasks  and  task  elements,  changes  in  procedures 
for  performing  tasks,  modification  of  performance  standards,  and 
decisions  that  affect  any  of  the  information  items  contained  in  the 
analysis.  To  accommodate  frequent  revisions  rapidly  and  economically, 
the  task  analysis  document  was  prepared  with  the  aid  of  the  HumRRO  IBM 
obO  computer  (a  sample  page  is  shown  in  Appendix  B) .  This  document 
presents,  in  a  computer  printed  format,  the  information  recorded  on  the 
forms  described  above.  The  identifying  numbers  of  the  tasks  and  task 
components  appear  in  the  left-hand  column  followed  by  the  task  descrip¬ 
tion.  The  Reference  column  contains  the  entries  relating  doctrine  to 
the  task  and/or  component.  Enabling  knowledges  are  described  in  the  next 
section,  followed  by  Column  A  which  contains  the  enabling  skill  category 
code  (.perceptual,  motor,  or  cognitive).  Column  B  contains  the  code 
referring  to  standards  of  performance  assigned  to  task  components  based 
upon  criticality  judgments,  live  course  location  entries  (page  number 
and  lesson  plan)  related  to  individual  task  components  are  found  under 
the  Column  C  heading.  Column  D  (Training  Objectives)  was  not  used  in  the 
present  study  but  will  be  discussed  in  Chapter  2. 

Whereas  task  analyses  have  been  performed  as  part  of  many  course 
development  efforts,  there  has  been  no  provision  for  matching  the 
analyses  and  course  content  systematically  as  has  been  attempted  in  this 
effort.  Quality  control  procedures  have  usually  been  limited  to  follow¬ 
up  procedures,  such  as  testing.  This  study  provided  an  opportunity  to 
bring  some  elements  of  quality  control  within  the  course  development 
operation  by  using  the  task  analysis  document  to  determine  that  items 
designated  as  training  objectives  were  in  fact  taught  within  the  course. 
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Chapter  2 

APPLICATION  OF  TRAINING  OBJECTIVES 


Use  of  the  task  analysis  to  control  content  of  the  7bA10  course 
assisted  the  curriculum  developers  in  arranging  the  training  objectives 
in  a  job-related  sequence.  lhe  curriculum  was  modified  with  regard  to 
instructional  content  and  form  and  sequence  of  presentation  utilizing 
the  functional  context  concept  of  course  development .  Changes  in 
sections  of  the  course,  based  on  the  task  analysis,  resulted  in  limiting 
instruction  in  enabling  knowledges  and  skills  to  that  essential  for  the 
Supplyman's  job,  deleting  many  hours  of  instruction  in  non-essential 
subjects,  increasing  the  amount  of  class  time  devoted  to  practical 
exercises,  and  confining  the  use  of  television  and  films  to  the  introduc¬ 
tion  of  instructional  blocks.  To  insure  agreement  between  the  task  list 
and  the  curriculum,  a  discrepancy  analysis  was  performed  to  identify 
task  list  elements  not  covered  in  the  training  and  training  content 
unrelated  to  tasks  in  the  list.  Such  differences  were  resolved  during 
the  final  revision  of  the  course,  prior  to  sending  it  to  the  training 
centers. 

A  functional  context  ( job-oriented)  approach  was  applied  to  the  course 
in  preference  to  a  subject-matter  oriented  approach,  because  it  would 
better  enable  New  Standards  personnel  to  learn  the  material.  The  purpose 
of  learning  new  material  frequently  is  unclear  to  students  being  trained 
by  a  conventional  approach,  for  they  must  retain  knowledge  over  an 
extended  period  before  it  is  applied  in  the  course.  The  learning  of 
novel,  unfamiliar  material  without  a  meaningful  context  can  be  unstimu¬ 
lating  and  actually  result  in  poor  motivation  for  learning1,  particularly 
among  marginal  students. 

FUNCTIONAL  CONTEXT  TRAINING 

Functional  context  training  is  a  method  of  sequencing  training  content 
in  which  the  intended  use  of  new  material  is  established  prior  to  the 
introduction  of  the  material  itself.  Establishing  for  the  student  the 
purpose  of  new  material  (J.e.,  providing  a  functional  context  for  it) 
generally  enhances  learning  by  creating  a  clear  need  for  the  material 
thus  providing  an  incentive  for  learning,  and  by  providing  the  student 
with  a  set  of  meaningful  associations  between  new  material  and  that  which 
he  has  learned  previously.  The  following  sequences  represent  applications 
of  the  functional  context  approach  to  the  7oA10  course. 


1  Shoemaker,  Marry  A.  i  *  VcjV.rr  ,V. 

IlumRRO  Professional  Paper  33-b7,  July  llJ(v  . 
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1.  Job  goal  to  job  procedure.  The  ultimate  goal  of  the  job  is 
introduced  before  the  specific  job  procedures  are  taught.  An  example 
would  be  that  of  providing  initially  a  summary  of  the  role  of  the  Army 
Supplyman,  then  introducing  material  dealing  with  the  specific  job 
procedures  in  the  context  of  the  total  job.  Thus,  U  H  0  supply  functions 
can  be,  when  introduced,  related  in  a  meaningful  manner  to  the  entire 
supply  system. 

2.  Practical  to  theoretical.  Instruction  begins  by  showing  the 
student  what  he  is  expected  to  do,  and  then  introducing  technical  or 
theoretical  content  as  needed  to  enable  him  to  do  it.  An  example  in  the 
7bAlO  course  is  the  instruction  in  arithmetical  operations  presented  at 
the  point  at  which  the  student  is  involved  in  their  utilization.  The 
instructional  material  is  tied  directly  to  the  performance  that  requires 
it . 

3.  Whole  to  part.  In  dealing  with  organizational  functions 
(e.g.,  duty  positions,  tasks),  instruction  begins  with  the  functions  of 
the  entire  unit  and  then  proceeds  to  the  functional  relationships  among 
its  components.  In  the  TbAlO  course,  the  duty  position  (e.g..  Unit  and 
Organization)  is  introduced,  the  relevant  tasks  are  outlined,  and  finally 
instruction  in  the  performance  of  the  components  of  these  tasks  is 
presented.  Subtasks  are  related  to  tasks,  and  tasks  to  duty  positions 
(parts  explained  in  terms  of  their  relation  to  the  whole). 

Since  the  task  list  ordered  activities  in  terms  of  their  job  sequence, 
its  use  in  preparing  training  content  facilitated  application  of  the 
functional  context  approach.  Some  instructors  and  technical  writers 
found  the  phrase  "teach  within  the  job  situation"  communicated  the 
essential  characteristics  of  the  functional  context  approach. 

COURSE  CONTENT 

New  instructional  content,  obtained  by  means  of  the  task  analysis, 
was  incorporated  into  the  curriculum  according  to  an  instructional 
sequence  that  followed  a  task  orientation  rather  than  subject-matter 
orientation.  This  did  not  preclude  using  occasional  blocks  of  enabling 
instruction  that  were  not  directly  related  to  a  specific  task  ie.g.,  a 
section  on  arithmetic).  However,  these  blocks  wore  relatively  small  and 
integrated  with  task-oriented  instruction. 

Working  copies  of  lesson  plans,  programs  of  instruction,  and  similar 
material  were  mailed  to  HumRRO  as  soon  as  they  were  prepared.  Comments  > 

were  returned  directly  to  the  individual  preparing  the  materials  with  a 
copy  sent  to  Headquarters,  Enlisted  Supply  Department.  As  tasks,  task 
components,  and  related  information  were  incorporated  by  QMS  personnel, 
this  information  was  fed  into  the  HumRRO  computer  and  revised  printouts 
were  furnished  to  QMS.  These  printouts  served  as  interim  updated  course 
control  documents. 
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CURRICULUM  REVISIONS 


Certain  sections  of  the  7bA10  course  were  added  or  deleted  as  a 
result  of  the  task  analysts'  work,  which  was  facilitated  by  close  inter¬ 
action  with  the  curriculum  developers.  Specific  changes  in  the  7bA10 
course  that  occurred  as  a  result  of  this  interaction  were  as  follows: 

1.  The  lecture  portion  of  the  course  was  reduced.  This  change 
was  based  on  a  directive  of  the  Project  100,000  program  itself.  Maximum 
classroom  time  was  given  to  practical  exercises  in  which  the  instructor's 
primary  role  was  to  guide  students  having  difficulty.  Information 
necessary  to  perform  the  exercises  was  presented  by  means  of  written 
materials  whenever  feasible. 

2.  Only  those  arithmetical  operations  actually  necessary  for 
the  Supplyman's  job  were  taught;  a  section  of  this  subcourse  dealing  with 
fractions  was  dropped. 

3.  The  use  of  television  and  films  was  confined  to  the  introduc¬ 
tion  of  instructional  blocks,  its  purpose  being  to  acquaint  the  student 
with  the  job  environment  in  which  his  procedural  instruction  would  be 
applied . 


4.  A  block  of  instruction  in  touch-typing  was  deleted  since  this 
level  of  proficiency  is  not  required  in  the  Supplyman's  job.  In  Its  place, 
familiarity  with  the  use  of  a  typewriter  was  provided  in  the  context  of 
instruction  on  the  preparation  of  request  forms  and  other  tasks  requiring 
this  activity. 

5.  By  adhering  closely  to  the  task  analysis  document,  50  hours  of 
instruction  in  non-essential  subject-matter  areas  were  dropped  from  the 
training  program.  The  deleted  material  included  a  two-hour  block  on  direct 
exchange  procedures,  all  instruction  on  subsistence,  POL's  and  the  stock 
locator  system.  The  section  on  organizational  clothing  annexes  was  dropped 
from  the  U  S  0  subcourse  because  this  task  is  infrequently  performed  by  the 
Supplyman. 

DISCREPANCY  ANALYSIS 

The  discrepancy  analysis  is  a  procedure  which  was  introduced  to  insure 
congruence  of  the  task  analysis  and  the  course  curriculum.  Although  this 
procedure  should  be  instituted  during  the  curriculum  development  process 
prior  to  course  evaluation,  the  final  evaluation  in  this  study  preceded 
the  discrepancy  analysis.  Implementation  of  the  activity  was  delayed  due 
to  requirements  to  produce  an  operational  course  in  time  to  accept  the 
scheduled  input  of  trainees  to  the  pilot  class.  However,  results  of  the 
discrepancy  analysis  were  utilized  in  the  final  ~bA10  course  revision. 

In  the  Course  Location  column  of  the  task  analysis  document,  the  page 
number  of  the  lesson  plan  dealing  with  each  specific  task  component  was 
indicated.  This  provided  a  means  of  comparing  training  content  with  the 
task  list  to  identify  task  elements  omitted  from  training  and,  equally 
important,  to  identify  training  content  which  was  not  related  to  any  task 
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in  the  listing.  The  results  of  this  process  were  communicated  to  QMS  in 
a  discrepancy  analysis  document,  which  also  had  a  form  designed  to  assist 
QMS  personnel  in  resolving  the  discrepancies. 

The  discrepancy  analysis  document  consisted  of  three  parts.  Part  1 
of  the  analysis  (Appendix  C,  page  )  was  a  list  of  those  procedures, 
tasks  and  task  elements  that  were  described  in  tiie  lesson  plans  but  not 
included  in  the  task  analysis.  (In  some  cases  an  item  may  have  appeared 
in  the  list  in  a  different  context,  but  not  have  been  mentioned  in  the 
particular  task  discussed  by  the  lesson  plan.)  For  reference  purposes, 
each  discrepant  item  was  given  a  code  number.  Following  the  description 
of  the  item,  its  course  location  (page  number  and  lesson  plan)  was 
indicated. 

Part  II  (Appendix  C,  page  )  was  a  list  of  procedures,  tasks  and 
task  elements  that  were  described  in  the  task  analysis  but  not  included 
in  the  lesson  plans.  (In  some  cases  an  item  may  have  been  covered  in  the 
course  in  a  different  context  but  not  mentioned  in  the  one  described  by 
the  task  list.)  Following  the  assigned  code  number  and  item  description, 
the  item's  task  list  identification  (location)  number  was  indicated.  The 
item  description  could  refer  to  one  task  element  or  to  all  the  components 
of  a  task  or  procedure. 

Part  III  was  a  form  provided  for  use  by  QMS  personnel  to  record  their 
decisions  and  corrections  regarding  the  discrepant  item.  It  consisted  of 
three  sections  (Appendix  C,  pages  ).  The  first  two  sections  listed 

the  code  numbers  of  the  discrepant  items.  Spaces  were  provided  to  indicate 
whether  the  item  would  be  added,  deleted,  or  remain  in  the  course  and/or 
task  list.  If  the  item  was  taught  in  a  different  context,  there  was 
enough  space  to  enter  the  relevant  page  and  lesson  plan  numbers.  If  the 
item  was  incorrectly  described,  corrected  statements  could  be  entered  on 
the  pages  provided  in  the  third  section  of  the  document. 

Most  of  the  discrepancies  were  minor.  However,  in  some  instances, 
the  omissions  uncovered  by  this  process  involved  important  instructional 
blocks  and  affected  the  end-of-course  test  results  (see  Chapter  3) .  The 
task  analysis  document  and  76A10  course  were  rev  is  d  to  reflect  the 
discrepancy  analysis  resolutions. 
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Chapter  3 
EVALUATION 


The  true  measure  of  a  training  program's  effectiveness  is  the  ability 
of  its  graduates  to  perform  effectively  on  the  job.  While  it  is  rarely 
feasible  to  assess  actual  job  performance  in  the  field,  a  sample  of  job 
tasks  may  be  assembled  into  an  end-ot'-course  performance  test  that 
closely  approximates  the  primary  job  requirements.  The  results  will 
provide  a  reasonable  estimate  of  job  ability  provided  (. a T  the  sample  of 
tasks  is  of  sufficient  size  and  is  sufficiently  representative  of  the 
job,  (.b)  test  conditions  realistically  approximate  the  essentials  of 
the  job  situation,  (.cl  test  performance  is  representative  of  t he  graduate's 
capability  (.e.g.,  no  cribbing),  and  |>1)  administrative  procedures  do  not 
compromise  the  fidelity  of  the  test. 

INTERIM  EVALUATION 

While  a  valid  assessment  of  the  revised  Supplvman  course  necessarily 
awaited  the  completion  of  the  task  analysis.  Headquarters,  USCONARC, 
requested  an  early  comparison  between  the  revised  Supplvman  course  and 
its  conventional  counterpart  as  administered  at  several  U.S.  Army  Training 
Centers  j.-VTC's),  Comparison  of  attrition  rates  for  the  first  pilot  class 
and  a  selected  7 1> A 1 0  course  at  Fort  McClelland,  conducted  by  QMS,  clearly 
favored  the  former  course,  whose  attrition  rate  was  zero.  However, 
differences  in  attrition  rates  can  reflect  differences  in  graduation 
standards  as  much  as  they  do  the  quality  of  instruction.  What  was  required 
was  an  assessment  of  the  proficiency  of  the  two  groups  in  order  to  assure 
that  the  reduction  in  attrition  was  not  achieved  at  the  expense  of  the 
trainee's  performance  capability. 

Method 

To  assess  the  proficiency  of  the  Supplvman  course  graduate,  a  job 
sample  performance  test  was  constructed  of  supply  tasks  selected  from  the 
duties'  description  of  t ho  Supplvman  MOS.  The  sample  consisted  of  lt> 
frequently  performed  supply  tasks  representing  a  wide  range  of  difficulty. 
Test  stations  were  set  up  in  which  tasks  were  re-created  as  realistically 
as  possible.  The  test  administrators,  playing  roles  of  various  supply 
personnel  such  as  customers,  suppliers,  or  supervisors,  established  the 
task  requirements  that  the  examinees  were  to  fulfill.  All  necessary 
materials  including  forms,  catalogues,  files,  and  manuals  were  provided. 

The  administrator  also  prompted  the  examinee  when  necessary,  making  a 
note  of  the  assistance  provided. 

Underlying  the  pret'onnance  of  supply  tasks  are  enabling  skills  and 
knowledges,  such  as  knowledge  of  supply  procedures ,  concepts,  and 


policies,  basic  arithmetic,  reading,  and  typing.  Tests  were  constructed 
to  tap  each  of  these  underlying  variables;  the  typing  test  was  of  the 
performance  variety  while  the  remainder  were  given  in  written  form. 

The  conventional  Supplyman  course  given  at  the  U.S.  Army  Training 
Center,  fort  Ord.  California,  was  selected  for  comparison  purposes 
because  of  the  availability  of  llumRRO  personnel  and  facilities  at  that 
installation.  It  was  necessary  to  select  graduates  from  four  fort  Ord 
classes  in  order  to  match  the  distribution  of  AI-'QT  scores  that  prevailed 
in  the  first  QMS  Supplyman  pilot  class.  A  total  of  1“  Category  IV  and 
2 3  Category  11-111  personnel  were  tested  at  each  of  the  two  installations. 

Resul ts 


Mean  scores  and  standard  deviations  for  the  performance  tests  and 
tests  of  enabling  skills  are  presented  in  Table  A.  Performance  tests 
were  scored  in  terms  of  the  following  four  variables: 

1.  Time  -  the  total  time  taken  to  complete  the  performance  test. 

2.  Critical  I'.rrors  -  The  number  of  errors  of  a  type  that  would 
result  in  serious  delays  or  incorrect  supply  items  being  received. 

5.  Xon-Cri  t  ical  terrors  -  The  number  of  errors  of  a  type  that 
would  result  in  minor  delays  or  incorrect  quantities  of  supply  items 
being  received. 

4.  Assistance  -  The  number  of  times  the  examinees  had  to  be 
prompted.  The  scores  for  the  arithmetic  and  supply  information  tests 
were  the  number  of  test  questions  answered  correctly.  Hie  reading  level 
refers  to  the  average  grade  level  equivalent  of  the  reading  test  score. 

The  typing  scores  are  based  on  an  arbitrary  weighting  of  time  and  errors. 

In  Table  3  the  results  are  presented  separated  for  the  QMS  Pilot  Class 
and  the  Fort  Ord  ATC  group.  Within  each  group,  scores  are  presented  for 
two  levels  of  ability  on  the  Armed  Forces  Qua  1  i  t'l cat  i on  Test  pAFQTl  . 
Category  11  and  III  versus  Category  U  personnel. 

Mental  Categories.  With  the  except  ion  of  non-critical  errors  and 
typing,  the  Category  11-111  personnel  scored  significantly  higher  on  all 
tests  than  their  Category  IV  classmates.  Apparently  the  non  critical 
errors  reflected  carelessness  more  than  lack  of  knowledge  and  were, 
therefore,  less  related  to  intellectual  differences.  Typing  places  more 
emphasis  on  perceptual -motor  skills  than  on  the  cognitive  ability  reflected 
in  the  AKQT  scores.  The  differences  between  Category  1\  and  Category  11- 
III  personnel  were  similar  in  the  QMS  course  and  the  ATC  courses. 

Performance  Tests.  The  pilot  group  required  much  more  time  and 
committed  more  critical  errors  while  the  ATC  group  committed  more  non- 
critical  errors  and  required  greater  assistance.  This  result  may  be 
partly  due  to  differences  in  emphasis  between  the  courses .  With  its 
practical  orientation,  the  pilot  course  provided  its  students  with 
considerable  practice  in  performing  tasks.  Consequent ly,  these  students 
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were  somewhat  more  self-sufficient  and  less  likely  to  make  simple 
mistakes  than  their  ATC  counterparts.  The  ACT  course,  because  of  its 
emphasis  upon  the  supply  system,  may  have  developed  in  its  students  a 
better  appreciation  for  the  consequences  of  error  and  enabled  them  to 
distinguish  and  avoid  critical  errors.  Also,  despite  the  equivalence  of 
the  two  groups  on  the  AFQT,  the  reading  level  of  the  ATC  group  averaged 
a  grade  higher.  This  might  also  account  for  the  ability  of  the  ACT 
group  to  work  somewhat  faster. 

Lest  much  confidence  be  placed  in  the  overall  results  of  this 
preliminary  study,  two  points  must  be  raised  immediately.  First,  since 
the  test  administrators  differed  between  the  two  installations,  the 
procedures  by  which  these  tests  were  administered  are  likely  to  have 
varied  despite  the  use  of  standard  instruction.  Observations  seem  to 
indicate  that  the  test  administrators  at  Fort  Lee  were  less  inclined  to 
record  all  of  the  assistance  they  provided  than  were  the  Fort  Ord 
administrators . 

The  second  reservation  concerning  the  overall  results  arises  from  the 
pattern  of  differences  in  individual  tasks,  as  shown  in  Table  4.  Only  on 
the  assistance  variable  was  there  a  consistent  pattern  for  the  entire 
test  (pilot  group  higher  than  ATC  group  on  all  tasks) .  On  the  remaining 
scoring  variables,  the  results  varied  from  task  to  task.  The  pilot  group 
appears  to  have  excelled  on  laundry  tasks,  requesting  supplies,  and 
receipt  of  undocumented  supplies.  The  ATC  group,  on  the  other  hand, 
appears  to  have  surpassed  the  pilot  group  on  normal  receipt  of  supplies. 
These  differences  are  presumably  related  to  the  content  and  emphasis  of 
instruction.  What  they  point  up  is  that  performance  on  the  overall  job 
sample  test  is  determined  by  the  way  the  sample  of  tasks  is  constituted 
to  form  a  test.  While  the  sample  was  a  fairly  representative  one,  it 
was  composed  of  only  16  tasks.  Some  other  sample  might  have  yielded 
slightly  different  overall  results. 

Enabling  Skills.  The  ATC  group  performed  significantly  better  on  the 
tests  of  reading  and  arithmetic  --  a  likely  reflection  of  the  skills  with 
which  they  entered  the  course  rather  than  those  acquired  as  a  part  of  the 
course.  The  ATC  group  also  performed  significantly  better  on  the  typing 
test.  In  this  case  the  difference  is  clearly  due  to  training.  The  ATC 
program  provided  51  hours  of  typing  instruction  throughout  the  program 
whereas  typing  in  the  pilot  course  was  a  concentrated  25-hour  block  early 
in  training.  The  difference  in  time  allotted  to  typing  instruction 
follows  from  differences  in  the  training  objectives  for  the  two  courses. 
The  objective  of  typing  instruction  in  the  pilot  course  was  to  provide 
only  sufficient  skill  to  enable  the  student  to  prepare  supply  documents. 
Touch  typing  and  the  typing  speed  that  it  developed  were  not  considered 
to  be  of  sufficient  value  to  the  Supplyman  in  his  assignment  to  warrant 
the  amount  of  practice  that  would  have  been  required. 

The  pilot  class  surpassed  the  ATC  group  by  a  wide  margin  on  the 
supply  knowledge  test.  That  the  test  was  a  difficult  one  is  evidenced  by 
the  small  number  of  items  correctly  answered.  The  superiority  of  the 
pilot  class  may  be  traced  to  the  consolidation  of  relevant  supply 
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Table  4 


PERFORMANCE  OF  PILOT  (QMS)  AND  ATC  CLASSES  ON  INDIVIDUAL  SUPPLY  TASKS" 


Task 

Time 

Error 

(Total) 

Error 

(Non- 

critical) 

Error 

(Critical) 

Assists 

1.  Ammo  requests 

P1 

0 

P 

0 

P 

2 .  Record  of  demand 

0 

0 

0 

0 

P 

3.  Title  insert 

0 

A 

P 

A 

P 

4.  Turn-in 

P 

* 

* 

* 

P 

5.  Turn-in 

A 

P 

P 

A 

P 

6.  Clothing  issue 

A 

A 

P 

A 

0 

7.  Clothing  issue 

0 

A 

0 

A 

P 

8.  Inventory 

0 

0 

*  * 

★  * 

0 

9.  Laundry  roster 

0 

P 

P 

P 

P 

10.  Laundry  list 

A 

P 

P 

P 

P 

11.  Receive  supplies 

P 

A 

A 

0 

P 

12.  Receive  supplies 

A 

0 

A 

0 

0 

13.  Receive  supplies 

A 

A 

A 

0 

0 

14.  Request  supplies 

P 

P 

P 

0 

P 

15.  Undocumented  supplies 

P 

P 

P 

0 

P 

16.  Consolidate  records 

A 

0 

0 

0 

0 

Total  tests 

A 

P 

P 

A 

P 

1 P  =  Pilot  group  scored  significantly  higher  (p  <  .051 
A  =  ATC  group  scored  significantly  higher  (p  <  .05) 

0  =>  No  significant  difference 
*  =  No  errors  committed 
**  =  Errors  not  scored  for  criticality 
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procedures  and  factual  in  format  ion  in  a  reference  text  that  was  furnished 
to  each  student.  While  a  reference  text  was  also  available  to  students 
of  the  ATC  course,  it  was  less  complete  with  respect  to  supply  procedures 
and  was  somewhat  less  readable  than  was  the  pilot  course  text  prepared 
specifically  for  the  lower  Al'QT  Category  IV  population. 

At t r i t ion .  The  test  results  reported  for  the  pilot  course  included 
all  of  the  students  who  entered  the  course  since  the  entry  class 
graduated.  However,  10  percent  of  the  students  in  classes  from  which  the 
ATC  sample  was  obtained  were  dropped  for  academic  reasons  prior  to  test¬ 
ing.  Had  these  poorer  students  been  included  in  the  (est  sample,  the 
performance  of  the  AIT!  group  obviously  would  have  been  lowered.  In 
addition  to  those  dropped,  almost  a  quarter  of  the  ATC  sample  had  been 
recycled  at  least  once.  None  of  the  pilot  course  students  had  been 
recycled.  It  is  reasonable  to  expect  that  a  similar  "extra"  week  or  more 
of  training  would  have  improved  to  some  extent  the  performance  of  the 
poorer  students  in  the  pilot  class. 

Summary 

The  results  of  the  interim  evaluation  are  of  questionable  significance 
in  comparing  the  two  training  programs  with  respect  to  the  proficiency  of 
their  graduates,  first  of  all,  the  goals  of  the  two  programs  differed, 
with  accompanying  differences  in  emphasis  given  aspects  of  training. 
Graduates  of  each  program  appeared  to  excel  in  different  aspects  of 
performance  and  on  different  tasks.  Secondly,  the  graduates  from  the 
pilot  course  differed  from  those  of  the  ATC  in  that  the  pilot  course 
graduates  had  never  been  recycled  and  represented  100  percent  of  the 
students  who  had  entered  the  course,  whereas  nearly  a  quarter  of  t  ho  ATC 
graduates  had  been  recycled  one  or  more  times  and  10  percent  of  the 
students  who  had  entered  the  course  had  been  dropped  before  graduation. 

A  third  limitation  to  comparing  the  effectiveness  of  the  pilot  and  ATC 
courses  on  the  basis  of  their  respective  graduates'  performance  on  this 
test  is  the  apparent  discrepancy  in  recording  assistance  rendered  by  the 
test  administ rators  at  the  two  locations. 

A  meaningful  comparison  would  require  (1)  the  students  of  each  course 
to  be  matched  in  A1:QT  scores,  to  be  starting  the  course  for  the  first 
time,  and  either  to  complete  the  course  without  interruption  or  to  be 
recycled  or  dropped  on  the  basis  of  objective,  standardised  criteria, 
and  (J)  strictly  standard  procedures  of  test  administration. 

It  should  be  noted,  however,  that  performance  on  this  test  was  not 
the  criterion  for  graduation.  The  standard  for  graduation  at  both 
locations  was  based  on  in -course  test  scores  and  instructor  judgment. 
Therefore,  the  reduction  in  attrition  achieved  by  the  pilot  course,  as 
compared  to  the  conventional  ATC  program,  indicates  that  the  pilot  course 
achieved  its  assigned  goal  of  better  accommodating  lower  Category  IV 
personnel . 
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FINAL  EVALUATION 


The  comparison  of  the  first  pilot  class  with  the  Alt'  group  established 
that  the  revised  Supplvman  course  was  more  effective  in  training  lower 
Category  IV  personnel.  It  did  not.  however,  establish  that  the  course 
could  produce  a  "qualified  Supplvman."  there  being  at  that  time  no 
explicit  standards  of  qual i ficat ion.  True,  the  student  had  to  "pass"  in 
order  to  graduate  --  but  a  passing  grade  represented  a  product  of  test 
scores  and  instructor  judgments  and  could  not  be  related  t o  a  specific 
level  of  potential  field  performance.  And  without  determination  of  the 
graduate's  ability  to  perform,  there  was  no  wax  of  answering  the 
fundamental  question  -  Can  the  lower  Category  IV  new  standards  inductee 
make  an  effective  contribution  to  the  militarv  effort  as  a  Supplvman'.' 

Method 

between  the  first  pilot  class  in  duly  l;H>‘  and  the  third  class  in 
February  l9o.X,  training  objectives  and  performance  standards  were 
generated  which  represented  the  best  estimate  ot'  supply  personnel  as  to 
what  a  qualified  Supplvman  should  be  able  to  do.  To  assess  the  degree 
to  which  objectives  and  standards  were  attained,  the  job  sample  perform 
unco  test  was  administered  to  graduates  of  the  third  pilot  class  in 
April  1  .  The  following  revisions  in  the  performance  test  were  made 

to  accommodate  changes  in  training  objectives  that  had  occurred  since 
the  first  pilot  class:  (11  four  of  the  original  tasks  were  deleted  as 
no  longer  appropriate;  (21  two  additional  stock  control  tasks  were 
included  because  of  the  expansion  of  responsibilities  in  this  area;  and 
(SI  two  pairs  of  tasks  from  the  original  test  were  combined  into  a  single 
task  due  to  their  consol idat ion  in  the  task  analysis.  The  product  of 
these  revisions  was  a  performance  test  composed  of  1.'  tasks,  10  of  which 
were  common  to  the  earlier  test.  A  minor  change  was  also  made  in 
administrative  procedures.  Since  the  original  training  objectives  of  the 
pilot  course  permitted  the  Supplvman  considerable  assistance  bv  his 
supervisor,  test  procedures  allowed  such  assistance  to  be  provided. 

However  the  final  task  analysis  had  separated  the  specific  responsibilities 
of  the  Supplvman  and  the  supervisor  in  these  "assistance"  tasks.  Therefore, 
on  the  final  test,  the  Supplvman  was  expected  to  perform  all  elements  of 
his  tasks  independently,  and  his  inability  to  do  so  was  recorded  as 
"failure"  --  the  same  as  completing  the  elements  incorrectly.  However, 
to  extract  maximum  information  from  the  test  situation,  an  examinee  who 
was  stumped  by  a  particular  element  was  prompted  and  permitted  tv' 
cont inue . 

Results 

Table  f*  shows  the  average  percentage  of  task  elements  passed  by  the 
third  Pilot  class  in  each  of  the  12  tasks.  Results  are  recorded 
separately  for  each  level  of  performance  i  .  .  .'.  ,S.  and  .;>)  .  \n 
asterisk  means  that  no  elements  of  the  task  in  question  had  performance 
standards  at  that  level. 
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Table  5 


AVERAGE  PERCENTAGE  OF  TASK  ELEMENTS  CORRECT 
FOR  EACH  CATEGORY  OF  PERFORMANCE 
STANDARD  —  THIRD  PILOT  CLASS 


Task 

Performance 

Standard 

.SO 

.70 

.80 

.90 

1. 

Prepare 

ammo  request  (O)1 

97 

89 

4k  ?- 

98 

-> 

Prepare 

repair  part  request  (01 

84 

78 

7o 

90 

3. 

Receive 

and  process  turn-in  (U) 

88 

* 

"4 

4. 

Process 

receipt  at  stock  control  (S) 

78 

08 

93 

90 

5. 

Issue  initial  clothing  (U) 

83 

88 

58 

91 

b , 

Prepare 

inventory  count  cards  (S) 

9S 

* 

88 

* 

7. 

Prepare  laundry  roster  and 
statement  (U) 

95 

A 

* 

95 

8. 

Prepare 

laundry  list  (U) 

100 

* 

* 

97 

9. 

Receive  normal,  over-  and  short- 
supplies  (S) 

85 

09 

7o 

70 

10, 

Request 

supplies  (0) 

80 

88 

* 

94 

11. 

Receive 

undocumented  supplies  (S) 

-T  > 

87 

75 

* 

12. 

Consolidate  clothing  and 
equipment  record  (U) 

49 

32 

A 

42 

Total  test 

83 

7o 

78 

85 

*0  =  Organization,  U  =  Unit,  S  =  Support 

2*  =  No  task  elements  classified  at  this  performance  standard 
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On  the  test  as  a  whole,  t he  class  passed  an  average  of  slightly  over 
80  percent  of  the  task  elements.  At  first  glance  this  looks  reasonably 
good;  however,  the  data  in  Table  5  show  that  there  was  no  relation 
between  the  examinee's  performance  and  the  criticality  of  the  task  elements 
as  indicated  by  the  reliability  requirements.  Students  performed  no  more 
reliably  on  the  critical  .8  and  .9  level  elements  titan  they  did  upon  the 
less  critical  .5  and  .7  elements.  The  consequence  of  this  uniform 
performance  in  the  face  of  varying  requirements  is  shown  in  Table  0. 

Whereas  everyone  met  the  .5  standards  on  the  test  (everyone  got  SOI 
or  more  of  the  .5  task  elements  in  the  test  correct),  only  eight  percent 
met  the  more  stringent  .9  standards.  Only  one  individual  in  the  entire 
class  met  each  of  the  standards.  If  meeting  the  performance  standards 
was  a  requirement  for  graduation,  only  this  one  individual  would  have 
graduated . 

Several  factors  were  responsible  for  the  failure  of  graduates  to 
meet  the  standards  in  the  more  critical  tasks,  first,  the  standards 
themselves,  which  must  be  treated  as  provisional,  represented  only  the 
best  estimate  of  supply  personnel  as  to  what  a  graduate  should  be  able  to 
do.  The  estimates  did  not  consider  what  would  be  needed  in  terms  of 
student  ability  and  training  to  achieve  the  standards.  Because  there  was 
no  trade-off  between  the  needs  and  the  resources  required  to  fulfill  them, 
the  standards  may  have  demanded  too  much  of  the  instructors  and  the 
students  within  the  available  course  time. 

A  second  factor  which  contributed  to  poor  student  performance  was 
the  time  pressure  which  prevented  full  application  of  the  analytic  data 
to  the  course  revision.  The  discrepancy  analysis,  described  in  an 
earlier  chapter,  revealed  many  cases  in  which  task  elements  or  even 
entire  tasks  were  inadvertently  omitted  from  lesson  plans.  The  effect  of 
these  discrepancies  is  evident  in  the  pattern  of  performance  over  the 
various  tasks.  The  poorest  performance  occurred  on  the  following  tasks: 
turn-in  (.Task  a),  receiving  normal,  over  and  under -suppl  ies  (Task  91, 
receiving  undocumented  supplies  (Task  111,  and  consol idat ing  organisa¬ 
tional  clothing  and  equipment  records  (Task  l.!l.  the  discrepancy  analysis 
disclosed  that  Task  1 J  was  not  covered  at  all,  task  .1  was  covered 
incompletely,  and  the  elements  of  Tasks  9  and  11  were  treated  solely  in 
the  context  of  other  tasks. 

The  mere  inclusion  of  a  particular  task  in  training  content  does  not 
assure  that  performance  standards  will  be  attained.  The  level  of 
performance  will  depend  upon  the  emphasis  given  to  the  task,  that  is, 
how  much  time  is  devoted  to  it  in  conference  sessions,  whether  it  is 
included  in  practical  exercises,  and  if  so,  how  many  times,  and  the 
importance  the  instructor  appears  to  attach  to  it  in  front  of  the  class. 

No  data  are  available  by  which  to  estimate  the  emphasis  given  to  each 
task.  However,  it  appears  that  the  instructors  received  the  information 
on  standards  too  late  to  utilize  them  fully  in  ('reparation  and  deliver) 
of  instruction. 


Table  6 


PERCENTAGE  OF  INDIVIDUALS  MEETING 
EACH  CATEGORY  OF  PERFORMANCE 
STANDARD  —  THIRD  PILOT  CLASS 


Task 

Performance 

Standard 

.50 

.70 

.80 

.90 

1. 

Prepare  ammo  request  (O)1 

100 

84 

*2 

96 

2. 

Prepare  repair  part  request  (0) 

100 

68 

76 

96 

3. 

Receive  and  process  turn-in  (U) 

96 

76 

it 

04 

4. 

Process  receipt  at  stock  control  (S) 

96 

56 

72 

36 

5. 

Issue  initial  clothing  (U) 

96 

92 

36 

48 

6. 

Prepare  inventory  count  cards  (S) 

96 

* 

68 

* 

7. 

Prepare  laundry  roster  and 
statement  (U) 

96 

* 

★ 

84 

8. 

Prepare  laundry  list  (U) 

100 

* 

* 

92 

9. 

Receive  normal,  over-  and  short- 
supplies  (S) 

96 

08 

76 

68 

10. 

Request  supplies  (0) 

100 

84 

* 

72 

11. 

Receive  undocumented  supplies  (S) 

72 

84 

24 

★ 

12. 

Consolidate  clothing  and 
equipment  record  (U) 

44 

00 

★ 

08 

Total  test 

100 

88 

52 

08 

*0  *  Organization,  U  =  Unit,  S  =  Support 

2*  ■  No  task  elements  classified  at  this  performance  standard 


22 


The  performance  of  the  third  pilot  class  cannot  legitimately  be 
compared  with  that  of  its  predecessor.  Not  only  were  the  objectives 
of  training  and  testing  different  in  the  two  cases,  but  the  students 
differed  markedly  in  ability.  The  AFQT  score  of  the  third  pilot  class  ' 
averaged  39  as  opposed  to  30  for  the  first  pilot  class.  Similarly, 
reading  level  averaged  over  a  grade  higher  for  the  third  class. 

Summary 

Training  objectives  and  performance  standards  were  developed  which 
represented  the  best  estimate  of  supply  experts  as  to  what  a  Supplyman 
course  graduate  should  be  able  to  do.  A  revised  job  sample  performance 
test  was  administered  to  graduates  of  the  third  pilot  class  in  April, 

1968,  to  assess  the  degree  to  which  these  objectives  were  attained. 

The  class  averaged  slightly  over  80  percent  correct  of  all  task 
elements  on  the  test.  However,  students  performed  no  more  reliably  on 
the  more  critical  items  (.8  and  .9  level  elements)  than  they  did  on  the 
.5  and  .7  elements.  Certain  factors  appeared  to  be  responsible  for 
graduates  failing  to  meet  the  standards  in  the  more  critical  areas.  One 
factor  may  have  been  performance  standards  that  were  too  demanding.  As 
they  represented  only  an  estimate  of  what  a  graduate  should  be  able  to  do, 
the  standards  should  be  treated  as  provisional.  Also,  the  factor  of  time 
pressure  undoubtedly  had  an  effect  on  the  application  of  task  analytic 
data  to  the  curriculum.  Based  upon  the  results  of  the  discrepancy  analysis, 
it  appears  that  instructors  received  certain  task  information  rather  late 
and  thus  were  uncertain  as  to  what  material  was  to  be  emphasized. 
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APPENDIX  C 

SAMPLE  PAGES  FROM  DISCREPANCY  ANALYSIS  DOCUMENT 

Course  Location 

Code  Procedure,  Task  or  Task  Component  Page  No. /Lesson  Plan 

1001  Procedure  for  using  AR  700-8400-1  to  obtain  02/B2 

authorized  allowances  of  individual  clothing.** 

1002  Enter  sizes  when  preparing  DA  Form  10-195.  04/B2 

1003  Check  DA  Form  10-195  against  AR  700-8400-1  when  06/B2 

validating  requests  for  individual  clothing  as 

initial  issue. 

1004  Enter  authorized  allowance  from  AR  of  item  07/B2 

requested  which  has  not  as  yet  b'en  issued  on 

DA  Form  10-195. 

1005  Leave  voucher  number  block  blank  when  preparing  07/B2 

DA  Form  3078. 

1006  Leave  size  and  cost  columns  blank  when  preparing  08/B2 

DA  Form  3078. 

1007  Entries  made  by  personnel  at  clothing  sales  store  08/B2 

on  DA  Form  3078. 

1008  Obtain  approval  from  Supply  Sergeant  or  Supply  12/B2 

Officer  to  prepare  DA  Form  3078  if  request  is 

valid  for  exchange  sale. 

1009  Prepare  DA  Form  3078  in  two  copies  for  exchange  12/B2 

sale. 

1010  Change  the  size  of  the  item,  if  necessary,  on  13/B2 

DA  Form  10-195  after  exchange  has  been  completed. 

1011  If  request  is  for  charge  sale,  refer  individual  14/B2 

to  Supply  Sergeant  and/or  the  Unit  Commander  to 

determine  if  a  charge  sale  issue  is  justified.  If 
their  approval  is  obtained,  prepare  request. 

1012  Disposition  of  DA  Form  3078  if  charge  sale.  Retain  14/B2 

one  copy  for  suspense  file  and  give  three  copies  to 
individual  to  take  to  clothing  sales  store. 

1013  Adjust  DA  Form  10-195  for  charge  sale  issue.  In  14/B2 

turn-in  column  give  "administrative  credit"  and 

in  issue  column,  "purchase  from  maintenance 
allowance". 

1014  Retain  completed  charge  sale  request  forms  in  unit  15/B2 

supply  files  for  one  year. 

1015  When  charge  sale  request  is  posted  to  DA  Form  10-195,  15/B2 

annotate  the  request  with  the  word,  "Posted". 


Task  List 
Identification 

Code  Procedure,  Task  or  Task  Component  Number 

2001  When  validating  a  request  received  for  exchange  1111130  ' 

of  individual  clothing,  check  if  individual  has 

medical  authorization  for  exchange. 

2002  When  validating  a  request  received  for  replace-  1111310-1111320 

ment  of  individual  clothing,  check  if  individual 

is  responsible  for  the  loss  or  damage,  and  if 
not  responsible,  record  an  explanation  of  the 
situation . 


2003  Procedure  for  validating  request  received  for  1120000-1121700 

organizational  clothing  and  equipment  at  unit 
and  at  organization. 


2004  When  validating  requests  received  for  repair  1141200-1141300 

parts,  if  item  is  on  PLL  and  in  stock,  issue 
item  and  make  request  for  replacement,  and  if 
not  in  stock  make  request  for  the  part. 


2005  Steps  to  follow  when  receiving  requests  for  1151000-1152000 

expendable  supplies  at  unit. 

2006  Steps  to  follow  when  receiving  requests  for  1153000-1154300 

expendable  supplies  at  organization  if  item 

on  hand  or  if  SSSC  available. 

2007  When  validating  unit  request  for  expendable  1155300 

supplies  at  organization,  deny  or  return 

request  if  it  is  not  valid. 


2008  Procedures  for  validating  requests  received  1160000-1161400 

for  non-expendable  supplies  at  unit  and 
organization . 


2009  When  making  a  request  for  a  charge  or  exchange  12115P0 

sale  of  individual  clothing,  if  the  request  is 

not  authorized,  deny  it  and  advise  individual 
to  purchase  item  at  own  expense. 

2010  When  making  request  for  gratuitous  issue  of  1211b20 

individual  clothing,  type  letter  for  unit  CO 

to  sign  if  required,  explaining  the  request. 
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Code 


1001 


1002 


1 


Will  continue  to  be 
taught  and  should  he 
included  in  task  list 


Will  he  dropp e d 
from  course. 


_ X 


ls  incorrect  as 
stated  but 
correction  will  be 
taught  and  should 
be  included  in 
task  list. 
(Indicate 
corrections  on 
_back  j>a£es . ) 


1003 


1004 


1005 


- - -X 


Code 


Will  be  taught  4 
inserted  in  lesson 
plans,  or  is  now 
taught  in  another 
context  4  will  be 
repeated  in  this  one. 
Should  remain  in 
task  list. 

(If  now  taught,  state 
page  number  4  lesson 
plan . ) _ 


Will  not  be  taught, 
is  incorrect,  or  is 
now  taught  in 
another  context  4 
will  not  be  repeated 
in  this  one. 

Should  be  dropped 
from  task  list. 

(If  now  taught,  state 
page  number  4  lesson 
plan . ) _ 


Is  incorrect  as 
stated  but 
correction  will 
be  taught  4  should 
be  included  in  task 
list . 

(Indicate 

corrections  on  back 
pages . ) 


